Feasibility of low-cost particle sensor types in long-term indoor air pollution health studies after repeated calibration, 2019–2021

Previous studies have explored using calibrated low-cost particulate matter (PM) sensors, but important research gaps remain regarding long-term performance and reliability. Evaluate longitudinal performance of low-cost particle sensors by measuring sensor performance changes over 2 years of use. 51 low-cost particle sensors (Airbeam 1 N = 29; Airbeam 2 N = 22) were calibrated four times over a 2-year timeframe between 2019 and 2021. Cigarette smoke-specific calibration curves for Airbeam 1 and 2 PM sensors were created by directly comparing simultaneous 1-min readings of a Thermo Scientific Personal DataRAM PDR-1500 unit with a 2.5 µm inlet. Inter-sensor variability in calibration coefficient was high, particularly in Airbeam 1 sensors at study initiation. Calibration coefficients for both sensor types trended downwards over time to < 1 at final calibration timepoint [Airbeam 1 Mean (SD) = 0.87 (0.20); Airbeam 2 Mean (SD) = 0.96 (0.27)]. We lost more Airbeam 1 sensors (N = 27 out of 56, failure rate 48.2%) than Airbeam 2 (N = 2 out of 24, failure rate 8.3%) due to electronics, battery, or data output issues. Evidence suggests degradation over time might depend more on particle sensor type, rather than individual usage. Repeated calibrations of low-cost particle sensors may increase confidence in reported PM levels in longitudinal indoor air pollution studies.


Methods
Generation of calibration curves for cigarette smoke. Cigarette smoke-specific calibration curves for the Airbeam 1 and 2 PM 2.5 sensors were created in a laboratory setting via the direct comparison of the output of the low cost Airbeam sensors with simultaneous 1-min readings produced by a factory-calibrated Thermo Scientific Personal DataRAM PDR-1500 unit with a 2.5 µm inlet (Thermo Environmental Instruments, Waltham, MA). The PDR-1500 unit is a widely used instrument and shown to be reliable from previous studies [21][22][23][24][25][26][27][28][29] . Over the course of the 2-year period, our low-cost sensors were calibrated four times using the same PDR-1500 unit. We took pre and post weight measurements of the internal filter within the PDR-1500 unit to calculate the gravimetric concentration which allowed for the calibration of the real-time readings. The Airbeam 1 and 2 devices utilize two low-cost sensors: The Shinyei PPD60PV and Plantower PMS 7003 infra-red light scattering particle sensors, respectively. The PDR 1500 unit was zeroed with particle-free air prior to each run.
To perform the calibration, 8-12 Airbeam units were placed into an airtight stainless-steel chamber, where temperature is room temperature and humidity matches the building's at below 50%, with access ports permitting the introduction of cigarette smoke or HEPA filtered air. The PDR-1500 was connected to a sampling port for measuring the PM 2.5 concentrations inside the chamber. This instrument has both an inlet and outlet where tubes are connected to inject cigarette smoke into the chamber; the PDR-1500 was not placed inside the chamber to prevent contamination resulting from its enclosure with cigarette smoke. A smoking machine (Borgwaldt, Hamburg, Germany) was used to inject fresh mainstream cigarette smoke using 3R4F reference cigarettes into the chamber until the PDR-1500 registered a particle mass concentration greater than 1000 µg/m 3 . A high concentration value such as 1000 µg/m 3 exceeds the upper limit for PM 2.5 values for both low-cost particle sensor types. Airbeam 1 and Airbeam 2 sensors have different saturation points at 80 µg/m 3 and 200 µg/m 3 , respectively (i.e., the light scattering derived PM 2.5 output plateaus), ensuring the decreasing PM 2.5 calibration curve would begin above their detection ceiling (approximately 180 µg/m 3 and 800 µg/m 3 , respectively). After cigarette smoke generation was stopped, the sample pump and internal filter of the PDR-1500 slowly removed cigarette smoke from the chamber which was replaced by HEPA-filtered room air. The resulting time-dependent decrease in PM 2.5 was used to develop the calibration curve. The start times of the Airbeam units and PDR-1500 particulate matter readings were synchronized, and the 1 min outputs were recorded beginning above the nominal upper detection limit and continued until the PDR-1500 values stabilized in the low single digit µg/m 3 range. Each run lasted approximately one hour.
Readings from each Airbeam (X-axis) were matched by synchronized timestamp with the corresponding values from the PDR-1500 (Y-axis). Using Excel, a unique calibration equation for each Airbeam unit was calculated by linear regression up to 80 µg/m 3 which was the expected upper limit for indoor PM 2.5 . Polynomial regression models were also generated; however, the output of these models was linear up to 80 µg/m 3 , which To assess the effect of particle composition on the calibration curve, the Airbeam devices were also calibrated using airborne particles in the NYC subway system. As in the cigarette smoke calibration procedure, the output of four Airbeam 1's and four Airbeam 2's was compared to the PDR 1500 PM 2.5 output and a calibration coefficient was calculated for subway PM 2.5 .
Field sampling periods. We calibrated 51 low-cost particle sensors (Airbeam 1 generation N = 29; Airbeam 2 generation N = 22) at 4 different timepoints over a 2-year period spanning from 2019 to 2021. After each laboratory calibration, the Airbeam units were deployed in a large, natural experiment evaluating the impact of new smoke-free housing (SFH) policies on air quality in public housing units every 6 months 18,30 . Due to the onset of the COVID-19 pandemic, we were unable to perform Airbeam sensor calibration from April-September 2020. A technician-based calibration error for select Airbeam 2 sensors only, from December-March 2021, led to their exclusion from data analysis at that timepoint. The final calibration timepoint was collected for all 51 Airbeam sensors from May-September 2021 to obtain a final calibration coefficient. Data analysis. We descriptively tabulated the mean (SD) calibration coefficients at four different 6-month timepoints over a 2-year period from 2019 to 2021 for the two different Airbeam sensor types. We performed independent t-tests to measure statistically significant differences in calibration coefficient means between particle sensor types, and characterized the between-and-within variability for calibration coefficient measurements. Because the light scattering properties of airborne particles are influenced by particle composition, we compared the mean (SD) calibration coefficients for cigarette smoke and subway PM 2.5 using an independent t-test. Lastly, we used a difference-in-difference (DID) approach to compare within-group changes between Airbeam 1 and Airbeam 2 sensors across four different calibration timepoints. Regression models included fixed effects for particle sensor type (Airbeam 1 vs Airbeam 2 sensors) and data collection timepoints (12,18,30 and 36 months post-SFH policy implementation 13 ). We adjusted for the clustering of individual Airbeam IDs and repeated measures overtime. Model-based mean differences with 95% confidence intervals were calculated for each particle sensor type over time. P-values were reported after implementation of the independent t-tests, with a significance level set at p < 0.05, using a two-sided test. All analyses were performed using SAS statistical software, version 9.4 (SAS institute).
We examined the individual time trends in calibration coefficient measurements for low-cost particle sensors over a 2-year period, grouped by particle sensor type (Supplemental Figure S1), and descriptively categorized all low-cost particle sensors that were taken out of circulation over the 2-year period (Supplemental Table S1). We then examined the correlation between the number of unique instances of use for individual Airbeam sensors, and their final calibration coefficients at the end of the 2-year period (Supplemental Table S2 and Supplemental Figure S2).

Results
Sample characteristics. We conducted a descriptive characterization of the mean (SD) calibration coefficients at four different timepoints over a 2-year timeframe from 2019 to 2021 (Table 1). At our first timepoint, our sample included a total of N = 56 Airbeam 1 sensors and N = 24 Airbeam 2 sensors. We observed more equipment failure over time in Airbeam 1 sensors (n = 27 out of 56, failure rate 48.2%) than in Airbeam 2 sensors (n = 2 out of 24, failure rate 8.3%). These equipment failures occurred for a variety of reasons including, but Calibration Coefficient = slope of the PDR−1500 (Y -axis) vs Airbeam (X -axis) calibration curve Table 1. Descriptive characterization of calibration coefficient measurements among two low-cost particle sensor types over a two-year timeframe, 2019-2021. SD standard deviation. www.nature.com/scientificreports/ not limited to cockroach infestations, not recording data properly (i.e., inconsistent relative humidity, temperature, or PM outputs), reading null values in PM measurements, and failure during calibration (Supplemental Table S1). As a result, our effective sample size decreased to N = 37 Airbeam 1 sensors and N = 21 Airbeam 2 sensors at the second timepoint, and N = 29 Airbeam 1 sensors and N = 22 Airbeam 2 sensors at the third and fourth timepoints. We thus restricted the analyses to the N = 29 Airbeam 1 sensors and N = 22 Airbeam 2 sensors available across all 4 calibration time points. We performed a secondary analysis to include all data points, both from units that performed well and from units that failed, and found no significant effect on outcome (see Supplement Table S3). The PM 2.5 concentration readout of Airbeam PM 2.5 sensors was less than that of the PDR-1500 reference instrument at each calibration timepoint.
Between-and-within variability in calibration coefficients for low-cost particle sensor types. On an individual unit basis, we observed a high degree of inter-sensor variability in calibration coefficients across both low-cost particle sensor types over a 2-year timeframe (Fig. 1). There was a notable decline in Airbeam calibration coefficients consistent across both low-cost particle sensor types, with values trending downward to below one at the final calibration timepoint. Inter-monitor variability was high in Airbeam 1 sensors at the first calibration timepoint and in Airbeam 2 sensors at the fourth calibration timepoint. During the second calibration timepoint, the mean (standard deviation (SD)) calibration coefficient for Airbeam   Comparison of calibration coefficients for cigarette smoke versus subway particulate matter. Because particle composition can affect light scattering properties, we characterized the comparison of calibration coefficient mean differences by particulate matter from two different sources at a single timepoint (

Correlation between unique instances of use and final calibration coefficient for individual low-cost particle sensor types.
To determine if sensor usage affected Airbeam output over time, we characterized the unique instances of use (i.e., the number of 7-day indoor sampling periods that a sensor was used), and the final calibration coefficient for all 51 individual Airbeam sensors (Supplemental Table S2). We examined the correlation between the number of 7-day indoor sampling periods that an individual sensor was used, and its final calibration coefficient at the fourth calibration timepoint (Supplemental Figure S2). We did not observe a strong correlation for Airbeam 1 sensors (R 2 = 0.16) or for Airbeam 2 sensors (R 2 = 0.09). The slope of the curve for Airbeam 1 suggests that the more the sensors were used, the greater the deviation of its output from the PDR-1500's output, while the Airbeam 2 curve suggested no change with an increase in usage.

Discussion
To our knowledge, this analysis is one of the first long-term longitudinal assessments of performance and reliability of low-cost particle sensors in measuring indoor tobacco smoking. We observed a high degree of inter-sensor variability across both particle sensor types, particularly in Airbeam 1 sensors at the study's initiation. Change Table 2. Characterization of the least square mean differences for each low-cost particle sensor type over a two-year timeframe, 2019-2021.  www.nature.com/scientificreports/ in calibration coefficients over time for individual Airbeam units was detected, suggesting a degradation of lowcost particle sensors for longitudinal assessment (Supplemental Figure S1). There were also notable downward trends in calibration coefficients over time, whereas the accompanying calibration coefficient for both Airbeam 1 and 2 sensors was below 1 at the final calibration timepoint. Findings lend support to the conclusion that the routine calibration of individual Airbeam units might help to improve their utility and performance over time. Overall, Airbeam 2 particle sensors fared better than Airbeam 1 sensors, suggesting greater durability of Airbeam 2 sensors for longitudinal assessment. Our findings suggest that low-cost particle sensors might be differentially subjected to degradation, seen in the greater loss of Airbeam 1 sensors than Airbeam 2 sensors over time. While two of these failures, and the loss of units, resulted directly from the public housing environments (i.e., roach infestation), other failures were more generally concerning for the use of non-calibrated low-cost particle sensors for longitudinal assessment of air quality. Interestingly, we did not observe a strong correlation between the unique instances of field use of sensors over the 2-year period and their final calibration coefficients measured at the fourth calibration timepoint, suggesting that low-cost sensor degradation over time might be more contingent on particle sensor type, rather than individual sensor usage.
Calibration coefficients differed modestly between cigarette smoke or subway PM (primarily combustion products and iron-rich friction particles, respectively), suggesting that the light-scattering physics of these lowcost particle sensors may slightly be affected by these two particle source types. Our finding, however, is limited to two particular particle types, and further studies are needed to assess calibration across a range of particles with different source-dependent compositions. Other researchers have observed that, in addition to particle composition, the accuracy of PM 2.5 sensor output also depends upon particle size and humidity 13 . Thus, low-cost sensors require routine calibration in the laboratory with the PM 2.5 and environmental conditions of interest.
Our current analysis provides a robust assessment of the longitudinal utility of low-cost particle sensors. Previous studies have measured the utility of low-cost particle sensors for PM monitoring where referencestandard equipment is not available or feasible, and for improving the study of spatially localized airborne PM concentrations [5][6][7][8][9][10][11][12][13][14] . One study conducted in the United Kingdom evaluated the performance of four models of low-cost PM sensors and examined inter-model performance across 19 different particle sensor units. Despite differences in the way each sensor type derived PM concentrations, the researchers found general agreement in PM readings across sensor types 8 . Another study evaluated the performance of two widely-used particle sensors, the Plantower PMS A003 and Shinyei PPD42NS, and developed PM calibration models for seven different metropolitan areas (i.e., Los Angeles, Chicago, New York, Baltimore, Minneapolis-St. Paul, Winston-Salem and Seattle) using a sample of 72 sensors. The authors found that good calibration models were feasible only with the Plantower PMS A003 model after running simulations for region-specific models 7 . Another study found that a Plantower PMS 1003 sensor provided reliable PM data outputs over a 13-month period 15 . Our study extended this time period to over 2 years of reliable output from Plantower PM sensors (albeit a different model), although the reproducibility of the calibration coefficients varied by individual units over time. One of the largest programs of low-cost sensor use is currently underway with the U.S. EPA's AirNow network of low-cost PurpleAir sensors for the nationwide monitoring of wildfire-generated PM (https:// www. airnow. gov/ fires/ using-airnow-duringwildfi res/). As demonstrated in our study of Airbeam sensors, the PurpleAir sensors report PM levels that differ from more expensive and reliable monitoring instruments, but these offsets can be corrected by a 'correction equation. ' The underlying design of the PurpleAir device is based on the fact that low cost sensors may degrade over time and therefore the PurpleAir device evaluates individual sensor degradation by continually comparing the output of two low-cost Plantower PM sensor units built into each monitoring device 31 . As such, the EPA has published guidelines on the use and performance testing of low-cost air pollution sensors (https:// cfpub. epa. gov/ si/ si_ public_ record_ Report. cfm? dirEn tryId= 35078 5& Lab= CEMM). Without such corrections, caution is necessary regarding the reliability of low-cost PM sensors over time.
There were several limitations to our research. Overall, the PM output of each low-cost particle sensor differed from the PM output of the widely used PDR-1500 which has an air flow regulation and infra-red laser that are far more precise than what is available in the low-cost PM sensors, suggesting a potential for under-or overestimation of PM levels when calibration methods are not utilized. Over time, we experienced equipment failures in a significant number of sensors, particularly the Airbeam 1 generation, thus reducing our effective sample size in this calibration study. The results in our paper may fail to cover all the low-cost sensors and calibration of lowcost PM sensors is imperative. Our routine calibration and inspection of low-cost particle sensors ensured careful use for the long-term sampling of indoor tobacco smoke. Unfortunately, this type of calibration would likely be challenging for many community groups or citizen science groups that may not have access to higher quality PM monitors. There were also several strengths to our research. Our study provides a robust assessment of the utility of low-cost particle sensors among a large number of a single brand of two generations of particle sensors available for purchase and utilized in citizen science across the U.S 32 . We compared the robustness of these two low-cost Airbeam particle sensor types, as well as across two different calibration particle types. We restricted our analysis to sensors that did not provide evidence of malfunction over time, and measured calibration coefficients over a 2-year period, allowing for the assessment of the reliability of these particles for air quality monitoring.

Conclusions
We observed modest changes in calibration coefficient measurements over a 2-year timeframe among both low-cost Airbeam particle sensor types, but in general the later generation Airbeam 2 model was more reliable, suggesting that specific particle sensors may yield better longitudinal consistency. While we did observe a degree of inter-monitor variability, changes in calibration coefficient measurements were relatively consistent across Airbeam 1 and 2 sensors. Finally, while not significant, we observed a modest difference in calibration coefficients